Consistent SDNs through Network State Fuzzing

The conventional wisdom is that a software-defined network (SDN) operates under the premise that the logically centralized control plane has an accurate representation of the actual data plane state. Unfortunately, bugs, misconfigurations, faults or attacks can introduce inconsistencies that undermine correct operation. Previous work in this area, however, lacks a holistic methodology to tackle this problem and thus, addresses only certain parts of the problem. Yet, the consistency of the overall system is only as good as its least consistent part. Motivated by an analogy of network consistency checking with program testing, we propose to add active probe-based network state fuzzing to our consistency check repertoire. Hereby, our system, PAZZ, combines production traffic with active probes to periodically test if the actual forwarding path and decision elements (on the data plane) correspond to the expected ones (on the control plane). Our insight is that active traffic covers the inconsistency cases beyond the ones identified by passive traffic. PAZZ prototype was built and evaluated on topologies of varying scale and complexity. Our results show that PAZZ requires minimal network resources to detect persistent data plane faults through fuzzing and localize them quickly while outperforming baseline approaches.Comment: Added three extra relevant references, the arXiv later was accepted in IEEE Transactions of Network and Service Management (TNSM), 2019 with the title "Towards Consistent SDNs: A Case for Network State Fuzzing

Runtime Verification of P4 Switches with Reinforcement Learning

© ACM 2019. This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in Proceedings of the 2019 Workshop on Network Meets AI & ML - NetAI’19, http://dx.doi.org/10.1145/3341216.3342206.We present the design and early implementation of p4rl, a system that uses reinforcement learning-guided fuzz testing to execute the verification of P4 switches automatically at runtime. p4rl system uses our novel user-friendly query language, p4q to conveniently specify the intended properties in simple conditional statements (if-else) and check the actual runtime behavior of the P4 switch against such properties. In p4rl, user-specified p4q queries with the control plane configuration, Agent, and the Reward System guide the fuzzing process to trigger runtime bugs automatically during Agent training. To illustrate the strength of p4rl, we developed and evaluated an early prototype of p4rl system that executes runtime verification of a P4 network device, e.g., L3 (Layer-3) switch. Our initial results are promising and show that p4rl automatically detects diverse bugs while outperforming the baseline approach.EC/H2020/679158/EU/Resolving the Tussle in the Internet: Mapping, Architecture, and Policy Making/ResolutioNetBMBF, 01IS17052, Adaptiver, Virtueller Assistent zur LAwinenwarnung Nach CHarakter Eigenschaften (AVALANCHE

Towards Transiently Secure Updates in Asynchronous SDNs

© ACM 2016. This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in Proceedings of the 2016 Conference on ACM SIGCOMM 2016 Conference - SIGCOMM ’16, http://dx.doi.org/10.1145/2934872.2959083.Software-Defined Networks (SDNs) promise to overcome the often complex and error-prone operation of tradi- tional computer networks, by enabling programmabil- ity, automation and verifiability. Yet, SDNs also in- troduce new challenges, for example due to the asyn- chronous communication channel between the logically centralized control platform and the switches in the data plane. In particular, the asynchronous commu- nication of network update commands (e.g., OpenFlow FlowMod messages) may lead to transient inconsisten- cies, such as loops or bypassed waypoints (e.g., fire- walls). One approach to ensure transient consistency even in asynchronous environments is to employ smart scheduling algorithms: algorithms which update subsets of switches in each communication round only, where each subset in itself guarantees consistency. In this demo, we show how to change routing policies in a transiently consistent manner. We demonstrate two al- gorithms, namely, Wayup [5] and Peacock [4], which partition the network updates sent from SDN controller towards OpenFlow software switches into multiple rounds as per respective algorithms. Later, the barrier mes- sages are utilized to ensure reliable network updates.EC/FP7/619609/EU/Unifying Cloud and Carrier Networks/UNIF

Auf dem Weg zur Laufzeitüberprüfung programmierbarer Netzwerke

Today, we rely heavily upon Internet-connected devices, applications, and services. Such reliance makes it crucial for the underlying networks to be reliable and secure. However, network outages, bugs in network devices such as switches and routers, and security compromises of the critical network infrastructure happen and sometimes, incur losses in revenue. This makes network management and control critically important. Networking devices such as IP-based routers and ethernet-based switches are fixed-function devices that are vertically integrated, i.e., the control and data plane are bundled together. These devices can only be configured by network operators and, thus, notably lag behind in terms of providing useful and custom abstractions for network management and orchestration. This necessitated the need for new network management and control paradigms. Software-defined Networking (SDN) emerged as a paradigm that introduced programmability. SDN decouples the previously distributed control plane from the data plane and outsources the control to a logically centralized software SDN controller running on commodity hardware. SDN controller manages and controls several data plane devices (e.g., SDN switches) through a well-known interface known as OpenFlow while offering a multitude of opportunities regarding management and control through programmability of the control plane. However, the SDN switches remain as fixed-function devices that can only be configured via OpenFlow but not programmed. To enhance SDNs, Programming Protocol-independent Packet Processors (P4) language was introduced. P4 is a domain-specific language that allows data plane programmability by writing custom programs known as P4 programs that run on data plane switches (e.g., P4 switches). These programs dictate P4 switches how to process packets even after those switches are already deployed. At the network level, a P4 network, generally, comprises of an SDN controller that manages and controls underlying P4 switches. Together, SDN and P4 are complementary and can operate in conjugation with each other. Hereby, programmable networks emerged as a generic term for networks such as Software-defined Networks (SDNs) or P4 networks. However, as the switch (e.g., a P4 switch) and the networks (e.g., SDNs and P4 networks) get increasingly programmable, complex bugs and faults surface especially, during runtime under diverse workloads to pose unprecedented threats. To give an example of a single P4 switch, a programmer writes a P4 program defining packet processing and then deploys it on a P4 switch. However, programming errors or P4 switch-specific errors manifesting as bugs result in an abnormal switch behavior under active traffic. This necessitates the detection and elimination of bugs. Zooming out from a P4 switch to the network-level (e.g., SDNs and P4 networks), data plane faults such as software or hardware bugs, switch failures, misconfigurations of switches, and attacks are ubiquitous. Even worse, the network control plane or SDN controller remains unaware of the existence of such data plane faults which results in control-data plane inconsistency. This necessitates sound mechanisms that verify the actual network data plane behavior against the expected network control plane behavior during runtime. To address the aforementioned challenges, the goal of this dissertation is to develop runtime verification mechanisms for the following scenarios ranging from a single switch to the network-level, namely, (i) a P4 switch, (ii) SDNs, and (iii) P4 networks. We make the following contributions: (i) We present the design, implementation, and evaluation of P6, a system that detects, localizes, and patches bugs at runtime in a P4 program deployed on a P4 switch to perform end-to-end runtime verification. P6 leverages techniques such as machine learning and static analysis-guided fuzzing to detect, dynamic fault localization-based techniques to localize, and software libraries to patch the bugs in P4 programs. Furthermore, P6 detects P4 switch target platform-specific bugs. Our evaluation of the P6 prototype on a single P4 switch across different switch targets shows that it successfully detects, localizes, and patches software bugs existing in the publicly-available P4 application programs while outperforming even the advanced baseline approaches. (ii) We present the design, implementation, and evaluation of PAZZ, a system that leverages network coverage-guided fuzzing via Binary Decision Diagrams (BDDs) for detection and bloom-filters used by custom per-switch tagging for localization of control-data plane inconsistency in SDNs. Our evaluation of the PAZZ prototype shows that it can quickly detect and localize the control-data plane inconsistency between one or many source-destination pairs in various topologies and configurations of different scale while outperforming baseline approaches. (iii) We present the design, implementation, and evaluation of P4CONSIST, a system that leverages In-band Network Telemetry (INT)-based per-switch tagging in P4 switches, depth-frst search, and symbolic execution for the detection of control-data plane inconsistency in P4 networks. Our evaluation of the P4CONSIST prototype shows that it can rapidly detect the control-data plane inconsistency between one or many source-destination pairs in diverse topologies and configurations of different scale.Heutzutage verlassen wir uns vermehrt auf Geräte, Anwendungen und Services, die mit dem Internet verbunden sind. Diese Abhängigkeit erfordert es, dass das zugrundeliegende Netzwerk zuverlässig und sicher ist. Netzwerkausfälle, Fehler in Netzwerkgeräten wie Switches und Router, sowie Beeinträchtigungen der Sicherheit von kritischen Netzwerkinfrastrukturen passieren und bringen Umsatzverluste mit sich. Aus diesem Grund sind Netzwerkmanage- ment und -kontrolle unerlässlich. Netzwerkgeräte wie IP-basierte Router und Ethernet-basierte Switches sind traditionell als vertikal integrierte Geräte mit fest verbauter Funktionalität, d.h. gebündelter Kontroll- und Datenebene, bekannt. Solche Geräte können durch Netzbetreiber lediglich konfiguriert werden, weshalb sie deutliche Nachteile im Bereich der Abstraktion von Netzwerkmanagement und -orchestrierung haben. Hieraus entstand die Notwendigkeit neuer Paradigmen für Netzwerkkontrolle und -management. Software-Defined Networking (SDN) entstand als Paradigma, das Programmierbarkeit von Netzwerken ermöglicht. SDN trennt die zuvor verteilte Kontrollebene von der Datenebene, die nun in einem logisch-zentralisierten SDN Controller konsolidiert wird und als Software auf Standardhardware ausgeführt werden kann. Der SDN Controller managt und kontrolliert mehrere Geräte auf Datenebene (z.B. SDN Switches), über eine als OpenFlow bekannte Schnittstelle und bietet gleichzeitig, durch die Programmierbarkeit der Kontrollebene, eine Reihe von Möglichkeiten. Nichtsdestotrotz verbleiben SDN Switches als Geräte mit fest verbauter Funktionalität, die lediglich konfiguriert, nicht aber programmiert werden können. Um die Programmierbarkeit zu ermöglichen, wurde die Program- miersprache Protocol-independent Packet Processors (P4) entwickelt. P4 ist eine domänen- spezifische Sprache, welche die Programmierbarkeit der Datenebene, durch das Entwickeln von benutzerdefinierten Programmen (sogenannten P4 Programmen), die auf Switches der Datenebene (z.B. P4 Switches) ausgeführt werden, ermöglicht. Diese Programme geben vor wie Pakete verarbeitet werden sollen, selbst wenn die P4 Switches bereits im Einsatz sind. Auf Netzwerkebene besteht ein P4 Netzwerk aus einem SDN Controller, der die untergeordneten P4 Switches managt und kontrolliert. Insgesamt sind SDN und P4 komplementär und können in Verbindung miteinander eingesetzt werden. Als Resultat entwickelte sich der Begriff programmierbare Netzwerke als Sammelbegriff, sowohl für Software-defined Networks (SDNs), als auch P4 Netzwerke. Mit der erhöhten Programmierbarkeit von Switches (z.B. P4 Switch), sowie Netzwerken (z.B. SDNs und P4 Netzwerke), treten insbesondere zur Ausführung unterschiedlicher Arbeitslasten komplexe Bugs und Fehler zu Tage, die bis dato nie dagewesene Bedrohungen darstellen. Um ein Beispiel für einen einzelnen P4 Switch zu nennen: Ein Programmierer schreibt ein P4 Programm, das die Paketverarbeitung definiert und stellt dieses dann auf einem P4 Switch bereit. Programmierfehler oder P4 Switch-spezifische Fehler manifestieren sich jedoch als Bugs, die in einem ungewöhnlichen Switch-Verhalten unter unterschiedlichem Datenverkehr resultieren. Sowohl die Erkennung als auch die Beseitigung von Bugs erlangen daher größte Bedeutung. Wenn der Fokus von einem P4 Switch auf die Netzwerkebene (z.B. SDNs und P4 Netzwerke) verlagert wird, zeigt sich, dass Fehler auf der Datenebene wie Software- oder Hardwarebugs, Hardwarefehler, Fehlkonfigurationen und Angriffe allgegenwärtig sind. Schlimmer noch: die Netzwerk-Kontrollebene oder der SDN Controller sind im Unklaren über die Existenz solcher Fehler auf Datenebene, was zu Inkonsistenzen zwischen Kontroll- und Datenebene führt während der Datenverkehr über das Netzwerk fließt. Dies erfordert solide Mechanismen, die das tatsächliche Verhalten der Datenebene mit dem erwarteten Verhalten der Kontrollebene während der Laufzeit vergleichen, um erforderliche Korrekturmaßnahmen einleiten zu können. Um die zuvor beschriebenen Herausforderungen zu adressieren, ist das Ziel dieser Dissertation die Entwicklung von Mechanismen zur Verifikation während der Laufzeit für die folgen den Szenarien, die von einem einzelnen Switch bis zur Netzwerkebene reichen, (i) einem P4 Switch, (ii) SDNs und (iii) P4 Netzwerke. Wir leisten folgende Beiträge: (i) Wir präsentieren das Design, die Implementierung und Evaluation von P6, einem System, welches Fehler in einem P4 Programm, ausgeführt auf einem P4 Switch, automatisch zur Laufzeit erkennen, lokalisieren und beheben kann, um Ende-zu-Ende Laufzeit Verifikation zur ermöglichen. P6 nutzt hierfür Techniken wie durch maschinelles Lernen und statische Analyse geleitetes Fuzzing zur Erkennung, dynamische Lokalisierungsmechanismen zur Lokalisierung und Softwarebibliotheken zur Behebung von Fehlern in P4 Programmen. Außerdem kann P6 plat- tformspezifische Fehler von P4 Switches erkennen. Unsere Evaluation des P6 Prototypen auf einem einzelnen P4 Switch mit einer Reihe verschiedener Zielplattformen zeigt, dass P6 verschiedenste existierende Fehler in öffentlich verfügbaren P4 Programmen, erfolgreich erkennen, lokalisieren und beheben kann und dabei sogar fortschrittliche Basisansätze übertrifft. (ii) Wir präsentieren das Design, die Implementierung und Evaluation von PAZZ, einem System zur Erkennung und Lokalisierung von Inkonsistenzen zwischen der Kontroll- und Datenebenen in SDNs, welches auf dem Netzwerkabdeckung basiertem Fuzzing mit Binary Decision Diagrams (BDDs) und Bloom Filtern zur Lokalisierung von Fehlern mittels benutzerdefinierter Markierungen von Paketen beruht. Unsere Evaluation des PAZZ Prototypen zeigt, dass er in der Lage ist, Inkonsistenzen der Kontroll- und Datenebene zwischen einem oder mehreren Quelle-Ziel Paaren in verschiedenen Topologien und Konfigurationen unter- schiedlicher Größenordnungen, schnell zu erkennen und zu lokalisieren, sowie die Basisansätze zu übertreffen. (iii) Wir präsentieren das Design, die Implementierung und Evaluation von P4CONSIST, einem System zur Erkennung von Inkonsistenzen zwischen der Kontroll- und Datenebene mithilfe von per-Switch Tagging basierend auf In-band Network Telemetry (INT) für P4 Switches, Tiefensuche und symbolischer Ausführung. Unsere Evaluation des P4CONSIST Prototypen zeigt, dass er in kürzester Zeit in der Lage ist, Inkonsistenzen zwischen der Kontroll- und Datenebene und einem oder mehrerer Quelle-Ziel Paaren in diversen Topologien und Konfigurationen unterschiedlicher Größenordnungen zu erkennen.BMBF, 01IS17052, Software Campus 2.0 (TU Berlin

No-hop: In-network Distributed Hash Tables

We make a case for a distributed hash table lookup in the network data plane. We argue that the lookup time performance of distributed hash tables can be further improved via an in-network data plane implementation. To this end, we introduce No-hop, an in-network distributed hash table implementation, which leverages the data plane programmability at line rate gained from P4. Our initial results of transporting distributed hash table logic from hosts' user space to the fast path of switches in the network data plane are promising. We show that No-hop improves the performance of locating the responsible host and maintains the properties of distributed hash tables while outperforming two baselines. Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.Cyber Securit

P4CONSIST: Towards Consistent P4 SDNs

The prevailing wisdom is that a software-defined network (SDN) operates under the premise that the logically centralized control plane has an accurate representation of the actual data plane state. Unfortunately, bugs, misconfigurations, faults or attacks can introduce inconsistencies between the network control and the data plane that can undermine the correct operation at runtime. Through our experiments, we realize that P4 SDNs are no exception, and are prone to similar problems. With the aim to verify the control-data plane inconsistency, we present the design and implementation of P4Consist, a system to detect the inconsistency between control and data plane in P4 SDNs. P4Consist generates active probe-based traffic continuously or periodically as an input to the P4 SDNs to check whether the actual behavior on the data plane corresponds to the expected control plane behavior. In P4Consist, the control plane and the data plane generate independent reports which are later, compared to verify the control-data plane consistency. The previous works in the field of monitoring and verification mostly aim to test the P4 programs through static analysis and thus, are insufficient to verify the network consistency at runtime. Experiments with our prototype implementation of P4Consist are promising and show that P4Consist can verify the control-data plane consistency in the complex datacenter 4-ary fat-tree (20 switches) and multipath grid (4, 9 and 16 switches) topologies with 60k rules per switch within a minimum time of 4 minutes. At the same time, P4Consist scales to multiple source-destination pairs to detect control-data plane inconsistency

Highlighting the Gap Between Expected and Actual Behavior in P4-enabled Networks

Modern networks increasingly rely on Software-defined Networking (SDN) and Network Function Virtualization (NFV) to augment their flexibility in high load scenarios. To further enhance the performance, a part of the functionality is often offloaded to forwarding devices, which are used as hardware accelerators and are configured by high level programming languages such as P4. However, hardware vendors use sophisticated technologies to implement these standards, which need to be understood by the programmer to avoid unintended behavior. In this demonstration we highlight the severe consequences of only relying on the network programming language when ignoring the device-specific limitations. We show this by the example of a Denial of Service attack against a P4-enabled SmartNIC. Finally, we discuss possible mitigations to this attack and stress the importance of an overall understanding of the entire system

Towards Consistent SDNS: A Case for Network State Fuzzing

The conventional wisdom is that a software-defined network (SDN) operates under the premise that the logically centralized control plane has an accurate representation of the actual data plane state. Unfortunately, bugs, misconfigurations, faults or attacks can introduce inconsistencies that undermine correct operation. Previous work in this area, however, lacks a holistic methodology to tackle this problem and thus, addresses only certain parts of the problem. Yet, the consistency of the overall system is only as good as its least consistent part. Motivated by an analogy of network consistency checking with program testing, we propose to add active probe-based network state fuzzing to our consistency check repertoire. Hereby, our system, PAZZ, combines production traffic with active probes to periodically test if the actual forwarding path and decision elements (on the data plane) correspond to the expected ones (on the control plane). Our insight is that active traffic covers the inconsistency cases beyond the ones identified by passive traffic. PAZZ prototype was built and evaluated on topologies of varying scale and complexity. Our results show that PAZZ requires minimal network resources to detect persistent data plane faults through fuzzing and localize them quickly while outperforming baseline approaches

Fix with P6: Verifying Programmable Switches at Runtime

We design, develop, and evaluate P6, an automated approach to (a) detect, (b) localize, and (c) patch software bugs in P4 programs. Bugs are reported via a violation of pre-specified expected behavior that is captured by P6. P6 is based on machine learning-guided fuzzing that tests P4 switch non-intrusively, i.e., without modifying the P4 program for detecting runtime bugs. This enables an automated and real-time localization and patching of bugs. We used a P6 prototype to detect and patch existing bugs in various publicly available P4 application programs deployed on two different switch platforms: behavioral model (bmv2) and Tofino. Our evaluation shows that P6 significantly outperforms bug detection baselines while generating fewer packets and patches bugs in large P4 programs such as switch.p4 without triggering any regressions.EC/H2020/679158/EU/Resolving the Tussle in the Internet: Mapping, Architecture, and Policy Making/ResolutioNetBMBF, 01IS17052, Software Campus 2.0 (TU Berlin